Display apparatus and method for controlling thereof

ABSTRACT

A display apparatus is disclosed. The display apparatus includes: a camera, a display, a memory storing an artificial intelligence model trained to identify a posture of a user based on location data with respect to a plurality of body parts of a user included in images and additional location data acquired based on the location data, and a processor configured to: control the display to display a training image and images photographed by the camera, identify the posture of the user included in the photographed images by inputting the location data with respect to the plurality of body parts of the user included in the photographed images, and control the display to display a training guide based on whether the posture of the user matches a posture corresponding to the training image.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based on and claims priority under 35 U.S.C. § 119to Korean Patent Application No. 10-2020-0122526, filed on Sep. 22,2020, in the

Korean Intellectual Property Office, the disclosure of which isincorporated by reference herein in its entirety.

BACKGROUND 1. Field

The disclosure relates to a display apparatus and a method forcontrolling thereof. For example, the disclosure relates to a displayapparatus configured to provide a training image, and a method forcontrolling thereof.

2. Description of Related Art

In recent years, the number of people who do exercises in a fitnesscenter for their health management, such as fitness training, yoga,Pilates, or the like is gradually increasing.

In addition, people who are unable to visit the fitness center due totime restrictions or want to exercise in a non-face-to-face manner areexercising through home training as an alternative manner.

Since home training is a way of exercising alone without a trainer, itis important for exercise effectiveness to accurately check whetherpeople are exercising in correct postures and to be provided with atraining guide for the exercise.

SUMMARY

Embodiments of the disclosure provide a display apparatus configured toidentify a user's posture using an artificial intelligence model, anddisplay a training guide based on whether the user's posture matches aposture of a training image, and a method for controlling thereof.

According to an example embodiment of the disclosure, a displayapparatus includes: a camera, a display, a memory storing an artificialintelligence module trained to identify a posture of a user based onlocation data with respect to a plurality of body parts of a userincluded in images, and additional location data acquired based on thelocation data, and a processor configured to: control the display todisplay a training image and images photographed by the camera, identifythe user's posture included in the photographed images by inputting thelocation data with respect to the plurality of body parts of the userincluded in the photographed images, and control the display to displaya training guide based on whether the user's posture matches a posturecorresponding to the training image.

The additional location data may be acquired by rotating the locationdata.

The processor may be configured, based on the user's posture natchingthe posture corresponding to the training image, to control the displayto display the training guide including information indicating that theuser's posture matches the posture corresponding to the training image,and based on the user's posture not matching the posture correspondingto the training image, to display the training guide includinginformation indicating that the user's posture does not match theposture corresponding to the training image.

The artificial intelligence model may be configured, based on locationdata with respect to a plurality of body parts of the user in a firstposture included in the images, and additional location data acquiredbased on the location data, to be trained to identify the first postureof the user, and wherein the first posture is configured to match theposture corresponding to the training image.

The artificial intelligence model may be configured to be trained toidentify a body part that has an exercise effect based on the firstposture, and wherein the processor is configured, based on the user'sposture being identified to match the posture corresponding to thetraining image, to control the display to display information on thebody part that has the exercise effect based on information on a bodypart output from the artificial intelligence model.

The artificial intelligence model may be configured to be trained toidentify a second posture of the user based on location data withrespect to a plurality of body parts of the user in a second postureincluded in the images, and additional location data acquired based onthe location data, and wherein the second posture does not match theposture corresponding to the training image.

The artificial intelligence model may be configured to be trained toidentify a body part that has a negative exercise effect based on thesecond posture, and wherein the processor is configured, based on theuser's posture not matching the posture corresponding to the trainingimage, to control the display to display information on a body part thathas the negative exercise effect based on information on a body partoutput from the artificial intelligence model.

According to an example embodiment of the disclosure, a method forcontrolling a display apparatus includes: displaying a training imageand images photographed by a camera, identifying a user's postureincluded in the photographed images by inputting a location data withrespect to a plurality of body parts of a user included in thephotographed images, and displaying a training guide based on whetherthe user's posture matches a posture corresponding to the trainingimage, wherein a artificial intelligence model is configured to betrained to identify the user's posture based on the location data withrespect to the plurality of body parts of the user and additionallocation data acquired based on the location data.

The additional location data may be acquired by rotating the locationdata.

The displaying may include, based on the user's posture matching theposture corresponding to the training image, displaying the trainingguide including information indicating that the user's posture matchesthe posture corresponding to the training image, and based on the user'sposture not matching the posture corresponding to the training image,displaying the training guide including information indicating that theuser's posture does not match the posture corresponding to the trainingimage.

The artificial intelligence model may be configured, based on locationdata with respect to a plurality of body parts of the user in a firstposture included in the images, and additional location data acquiredbased on the location data, to be trained to identify the first postureof the user, and wherein the first posture matches the posturecorresponding to the training image.

The artificial intelligence model may be configured to be trained toidentify a body part that has an exercise effect based on the firstposture, and wherein the displaying includes, based on the user'sposture matching the posture corresponding to the training image,displaying information on the body part that has the exercise effectbased on information on a body part output from the artificialintelligence model.

The artificial intelligence model may be configured to be trained toidentify a second posture of the user based on location data withrespect to a plurality of body parts of the user in a second postureincluded in the images, and additional location data acquired based onthe location data, and wherein the second posture does not match theposture corresponding to the training image.

The artificial intelligence model may be configured to be trained toidentify a body part that has a negative exercise effect based on thesecond posture, and wherein the displaying includes, based on the user'sposture not matching the posture corresponding to the training image,displaying information on a body part that has the negative exerciseeffect based on information on a body part output from the artificialintelligence model.

According to various example embodiments of the disclosure, anartificial intelligence model for identifying the user's posture istrained based on location data and additional location data acquired byrotating the location data. Even if the user is photographed fromvarious angles, it is possible to more accurately identify the user'sposture included in the image.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features and advantages of certainembodiments of the present disclosure will be more apparent from thefollowing detailed description, taken in conjunction with theaccompanying drawings, in which:

FIG. 1 is a diagram illustrating an example display apparatus accordingto various embodiments;

FIG. 2 is a block diagram illustrating an example configuration of adisplay apparatus according to various embodiments;

FIGS. 3 and 4 are diagrams illustrating an example process of trainingan artificial intelligence model according to various embodiments;

FIGS. 5A and 5B are diagrams illustrating an example method of providinga training guide in a display apparatus according to variousembodiments;

FIGS. 6A and 6B are diagrams illustrating an example method of providinga training guide in a display apparatus according to variousembodiments;

FIG. 7 is a diagram illustrating an example method of providing atraining guide in a display apparatus according to various embodiments;

FIGS. 8A and 8B are diagrams illustrating an example method of providinga training guide in a display apparatus according to variousembodiments;

FIG. 9 is a block diagram illustrating an example configuration of adisplay apparatus according to various embodiments; and

FIG. 10 is a flowchart illustrating an example method of controlling adisplay apparatus according to various embodiments.

DETAILED DESCRIPTION

The above and other aspects, features, and advantages of certain exampleembodiments of the disclosure will be more apparent from the followingdescription taken in conjunction with the accompanying drawings.However, it should be understood that the disclosure is not limited tothe specific embodiments described hereinafter, but includes variousmodifications, equivalents, and/or alternatives of the embodiments ofthe disclosure. In relation to explanation of the drawings, similardrawing reference numerals may be used for similar elements.

In the disclosure, the terms “include”, “may include”, “comprise” or“may comprise” designate the presence of features, numbers, steps,operations, components, elements, or a combination thereof that arewritten in the specification, but do not exclude the presence orpossibility of addition of one or more other features, numbers, steps,operations, components, elements, or a combination thereof.

In the description, the term “A or B”, “at least one of A or/and B”, or“one or more of A or/and B” may include all possible combinations of theitems that are enumerated together. For example, the term “A or B” or“at least one of A or/and B” may designate (1) at least one A, (2) atleast one B, or (3) both at least one A and at least one B.

The expression “1”, “2”, “first”, or “second” as used herein may modifya variety of elements, irrespective of order and/or importance thereof,and to distinguish one element from another, without limiting thecorresponding elements.

When an element (e.g., a first element) is “operatively orcommunicatively coupled with/to” or “connected to” another element(e.g., a second element), an element may be directly coupled withanother element or may be coupled through the other element (e.g., athird element). On the other hand, when an element (e.g., a firstelement) is “directly coupled with/to” or “directly connected to”another element (e.g., a second element), an element (e.g., a thirdelement) may not be provided between the other element.

In the disclosure, a ‘module’ or a ‘unit’ performs at least one functionor operation and may be implemented by hardware or software or acombination of the hardware and the software. In addition, a pluralityof ‘modules’ or a plurality of ‘units’ may be integrated into at leastone module and may be at least one processor except for ‘modules’ or‘units’ that should be realized in a specific hardware.

In the description, the term “configured to” may be used interchangeablywith, for example, “suitable for”, “having the capacity to”, “designedto”, “adapted to”, “made to”, or “capable of” under certaincircumstances. The term “configured to (set to)” does not necessarilyrefer to “specifically designed to” in a hardware level. Under certaincircumstances, the term “device configured to” may refer to “devicecapable of” doing something together with another device or components.For example, a phrase “a sub-processor configured to (set to) perform A,B, and C” may refer to a generic-purpose processor (e.g., CPU orapplication processor) capable of performing corresponding operations byexecuting a dedicated processor (e.g., embedded processor) forperforming corresponding operation, or executing one or more softwareprograms stored in a memory device.

FIG. 1 is a diagram illustrating an example display apparatus accordingto various embodiments.

The display apparatus 100 may perform a function for home training of auser.

In other words, the display apparatus 100 may display a training image,for example, a demonstration image of a trainer, on one area of thedisplay, and also display an image photographed by a camera of thedisplay apparatus 100 on another area of the display.

Accordingly, the user may check his or her posture (or exercise posture)while following the trainer's demonstration posture in the trainingimage.

The display apparatus 100 may identify whether the user's posturematches the posture in the training image and provide a training guidefor the user's posture.

The training guide may include information on whether the user's posturematches or does not match the trainer's posture.

For this operation, the display apparatus 100 may identify the user'sposture using an artificial intelligence model. The artificialintelligence model may include a model trained to identify the user'sposture from a corresponding image even if the user is photographed fromvarious angles.

Accordingly, the display apparatus 100 according to an embodiment maymore accurately identify the user's posture, which will be described ingreater detail below.

FIG. 2 is a block diagram illustrating an example configuration of adisplay apparatus according to various embodiments.

Referring to FIG. 2, the display apparatus 100 may include a camera 110,a display 120, a memory 130, and a processor (e.g., including processingcircuitry) 140.

The camera 110 may photograph images. For example, the camera 110 mayacquire an image by photographing a front side of the display apparatus100.

For this operation, the camera 110 may include an image sensor forreceiving external light, and an image photographed through the imagesensor may be acquired.

The display 120 may display an image. In this case, the display 110 maybe implemented as various types of displays such as, for example, andwithout limitation, LCD, LED, OLED, or the like.

The memory 130 may store various commands, programs or data necessaryfor the operation of the display apparatus 100.

The memory 130 may be implemented as a non-volatile memory, a volatilememory, a flash memory, a hard disk drive (HDD) or a solid state drive(SDD). The memory 130 may be accessed by the processor 140, and performreadout, recording, correction, deletion, update, and the like, on databy the processor140.

For example, the memory 130 may include an artificial intelligence model(e.g., including executable program elements executed by the processor)131 trained to identify the user's posture included in the image.

For this operation, the artificial intelligence model 131 may beimplemented as a classifier for identifying the user's posture includedin the image. In this case, the artificial intelligence model 131 mayoutput a probability that the user's posture corresponds to a specificposture.

An example process in which the artificial intelligence model is trainedwill be described in greater detail below with reference to FIGS. 3 and4.

The artificial intelligence model 131 may be a model trained to identifythe user's posture based on location data for a plurality of body partsof the user included in images and additional location data acquiredbased on the location data.

The images are training images for training the artificial intelligencemodel 131, and the user included in the images may be a user differentfrom the user who performs home training

In addition, the location data for the plurality of body parts mayinclude coordinate data for the plurality of body parts, and may includea plurality of key points indicating a plurality of joint parts of theuser included in the image.

In order to obtain location data, an artificial intelligence model (notshown) for extracting a point of a human joint from the image may beused. However, this is only an example, and location data may beacquired in various ways.

In addition, the additional location data may be data acquired byrotating the location data.

For example, the plurality of key points acquired from the image may betwo-dimensional coordinates. In this case, a plurality ofthree-dimensional coordinates may be acquired by converting the 2Dcoordinates to three-dimensional coordinates (3D) and applying randomrotational transformation to the 3D coordinates, and additional locationdata may be acquired by converting the plurality of 3D coordinates tothe 2D coordinates.

A method of acquiring additional location data according to anembodiment of the disclosure will be described in greater detail.

Depths for the plurality of key points acquired from the image may beestimated, and the plurality of key points may be transformed into 3Dusing this.

For example, an expected length for each body part may be calculatedthrough a ratio of a size of each body part to a total size of the userincluded in the image.

For example, if the length of each body part is predefined according tothe ratio of the size of each body part to the total size, the expectedlength of each body part may be calculated through the ratio of the sizeof each body part to the total size of the user included in the image.

The body part may be a body part defined by key points. For example, ifthe key points are wrist and elbow, the body part may be arm partbetween the wrist and the elbow defined by these key points.

A depth value for one of the plurality of key points, for example, a zvalue may be set to 0, and a depth value for a key point adjacentthereto may be calculated.

For example, if the z value is set to 0 for a key point (x₁, y₁), 3Dcoordinates of the key point may be (x₁, y₁, z₁) (here, z₁=0). In thiscase, when L is an estimated distance of the body part defined by thekey point (x₁, y₁) and the key point (x₂, y₂) adjacent key point (x₁,y₁), a depth value z2 of the adjacent key point (x₂, y₂) may becalculated based on Equation 1.

z ₂ =z ₁+√{square root over (L ²−(x ₁ −x ₂)²−(y ₁ −y ₂)²)}  [Equation 1]

Accordingly, the key points (x₁, y₁) and (x₂, y₂) may be converted into3D coordinates such as (x₁, y₁, z₁) and (x₂, y₂, z₂), respectively.Through this method, the z value of the key point adjacent thereto may,for example, be sequentially calculated based on the key point fromwhich the z value is calculated.

Accordingly, each of the plurality of key points may be transformed into3D.

A plurality of rotated 3D key points may be acquired by applying arandom rotation transformation to the plurality of 3D key points.

For example, random rotation transformation may be applied to each ofthe plurality of 3D key points based on Equation 2 below.

$\begin{matrix}{\begin{bmatrix}x^{\prime} \\y^{\prime} \\z^{\prime}\end{bmatrix} = {{\begin{bmatrix}{\cos({ay})} & 0 & {\sin({ay})} \\0 & 1 & 0 \\{- {\sin({ay})}} & 0 & {\cos({ay})}\end{bmatrix}\begin{bmatrix}1 & 0 & 1 \\0 & {\cos({ax})} & {- {\sin({ax})}} \\0 & {\sin({ax})} & {\cos({ax})}\end{bmatrix}}{\quad{\begin{bmatrix}{\cos({az})} & {- {\sin({az})}} & 1 \\{\sin({az})} & {\cos({az})} & 0 \\1 & 0 & {\cos({az})}\end{bmatrix}\begin{bmatrix}x \\y \\z\end{bmatrix}}}}} & \left\lbrack {{Equation}\mspace{14mu} 1} \right\rbrack\end{matrix}$

Here, zy, ax, az may represent randomly selected angle, (x, y, z) mayrepresent a 3D key point, and (x′, y′, z′) may represent a 3D key pointto which random rotation transformation is applied, respectively.

Accordingly, a rotated 3D key point may be acquired with respect to eachof the plurality of 3D key points.

Additional locational data may be acquired by converting the pluralityof rotated 3D key points into 2D.

For example, a 2D key point such as (x′, y′) may be acquired byprojecting each of the plurality of rotated 3D key points (x′, y′, z′)in 2D.

Accordingly, a plurality of 2D key points in which the plurality ofrotated 3D key points are converted into 2D may be acquired, andaccordingly, additional location data including the plurality of 2D keypoints may be acquired.

Images may be images photographed by a user performing an exercise in afirst posture or a second posture.

The artificial intelligence model 131 may be trained to identify theuser's first posture based on the additional location data acquiredbased on the location data with respect to the user's plurality of bodyparts in the first posture included in the images and the additionallocation data acquired based on the location data.

The first posture may be a posture that matches a posture correspondingto the training image.

For example, the first posture may be the same posture as the trainerincluded in the training image, and may be a posture in which directionsand angles of face, neck, torso, arms, legs, etc. are within apredetermined threshold range from a demonstration posture of thetrainer.

In other words, the artificial intelligence model 131 may be trained toidentify the user's posture matched with the training image using imagesphotographed by a user exercising with an accurate posture.

For example, referring to FIG. 3, location data 311 may be acquired fromimages 310 of the user performing “squat”, and additional location data312 may be acquired based on the location data.

In this case, the artificial intelligence model 131 may be trained usingthe location data 311 and the additional location data 312 as inputdata, and the “squat” posture as output data.

Similarly, the artificial intelligence model 131 may be trained using alocation data 321 acquired from images 320 in which a user performing“lunge” is photographed, and additional location data 322 acquired basedon the location data 321 as input data, and using the “lunge” posture asoutput data. In addition, the artificial intelligence model 131 may betrained using a location data 331 acquired from images 330 in which auser performing “plank” is photographed, and additional location data332 acquired based on location data 331 as input data, and using the“plank” posture as output data

As such, the artificial intelligence model 131 may be trained, for eachof a plurality of postures, using location data acquired from images inwhich a user is photographed performing each posture, and additionallocation data acquired based on the location data.

Accordingly, the artificial intelligence model 131 may be trained tooutput the user's posture included in the image when location data for aplurality of body parts of the user acquired from an image in which theuser is photographed.

According to an embodiment, the artificial intelligence model 131 may betrained to identify a body part where an exercise effect appearsaccording to the first posture.

In other words, the artificial intelligence model 131 may be trainedusing not only location data and additional location data, but alsoinformation on body parts that are effective according to the user'sposture as input data.

For example, a “squat” may be an effective exercise for legs, hips, andinner thighs.

In this case, the artificial intelligence model 131 may be trained usingnot only the location data and additional location data for the “squat”,but also information about legs, hips, and thighs as input data, and the“squat” posture and the “legs, hips, and inner thighs” as output data.

Accordingly, when the location data for a plurality of body parts of theuser acquired from the image of the user is input, the artificialintelligence model 131 may be trained to output information about thebody parts that have an effect in the corresponding posture.

The artificial intelligence model 131 may be trained to identify theuser's second posture based on the location data of the plurality ofbody parts of the user in the second posture included in the images andthe additional location data acquired based on the location data.

The second posture may be a posture that does not match a posturecorresponding to the training image.

For example, the second posture may be the same as the trainer'sdemonstration posture included in the training video, but its directionand angle of the face, neck, torso, arms, legs, etc. may be outside thetrainer's demonstration posture and a predetermined threshold range.

In other words, the artificial intelligence model 131 may be trained toidentify the user's posture that does not match the training image usingimages photographed by a user exercising in an incorrect posture.

For example, referring to FIG. 4, location data 411 may be acquired fromimages 410 of the user performing “squat” in an incorrect posture, andadditional location data 412 may be acquired based on the location data.

In this case, the artificial intelligence model 131 may be trained usingthe location data 411 and the additional location data 412 as input dataand the “incorrect squat” posture as output data.

Similarly, the artificial intelligence model 131 may be trained usinglocation data 421 acquired from images 420 of the user performing“lunge” in an incorrect posture and additional location data 422acquired based on the location data 421 as input data, and “incorrectlunge” posture as output data. In addition, the artificial intelligencemodel 131 may be trained using location data 431 acquired from images430 of the user performing “plank” in an incorrect posture andadditional location data 432 acquired based on the location data 431 asoutput data.

As such, the artificial intelligence model 131 may be trained for eachof a plurality of postures, using the location data acquired from imagesof the user is in an incorrect posture, and additional location dataacquired based on the location data.

Accordingly, the artificial intelligence model 131 may be trained tooutput an incorrect posture of the user included in the image whenlocation data for the plurality of body parts of the user acquired froman image of the user is input.

According to an embodiment, the artificial intelligence model 131 may betrained to identify a body part that have a negative exercise effectaccording to the second posture.

In other words, the artificial intelligence model 131 may be trainedusing not only location data and additional location data, but alsoinformation on body parts that have a bad effect according to the user'sincorrect posture as input data.

For example, the “inaccurate lunge” posture may strain the user's waist.In this case, the artificial intelligence model 131 may be trained usingnot only location data and additional location data for “inaccuratelunge”, but also information about the waist as input data, and the“squat” posture and “waist” as output data.

Accordingly, when location data for a plurality of body parts of theuser acquired from the image of the user is input, the artificialintelligence model 131 may be trained to output information about theincorrect posture of the user included in the image and body parts thathave negative exercise effect in the corresponding posture.

As described above, according to an embodiment of the disclosure, sincethe artificial intelligence model 131 is trained based on location dataand additional location data acquired by rotating the location data,even if the user is photographed from various angles, the artificialintelligence model 131 may identify the user's posture included in thecorresponding image.

Accordingly, according to an embodiment of the disclosure, even if theuser performs an exercise according to a training image at a locationconvenient for the user according to environmental circumstances, not infront of the display apparatus 100, the user's posture may be accuratelyidentified.

The training of the artificial intelligence model 131 described abovemay be performed by a server (not shown), and may be stored in thememory 130 when the display apparatus 100 is manufactured, and may bestored in the memory 130 by being downloaded from the server (notshown). Also, such training may be performed by the processor 140.

The processor 140 may include various processing circuitry and beelectrically connected to the camera 110, the display 120, and thememory 130 to control overall operations and functions of the displayapparatus 100.

The processor 140 may include, for example, and without limitation, acentral processing unit (CPU), a dedicated processor, an applicationprocessor (AP), or the like, and may execute one or more softwareprograms stored in the memory 130 according to one or more instructionsstored in the memory 130.

The processor 140 may control the display 110 to display a trainingimage and an image photographed by the camera 110.

The training image may include a demonstration image of a trainer. Inthis case, the training image may be stored in the memory 130 orprovided to the display apparatus 100 through streaming.

For example, when a user command for executing a training application isinput, the processor 140 may execute the training application stored inthe memory 130. The training application may be stored in the memory 130when the display apparatus 100 is manufactured, and be also downloadedwith the application from a server (not shown) providing variousapplications and stored in the memory 130.

The processor 140 may control the display to display the training imageon an area of the display 110. The processor 140 may control the camera110 photograph a front of the display apparatus 100 and display thephotographed image on another area of the display 110.

In addition, the processor 140 may identify the user's posture includedin the photographed image by inputting location data on a plurality ofbody parts of the user included in the photographed image into theartificial intelligence model 131, and identify whether the user'sposture matches the posture corresponding to the training image.

The location data for the plurality of body parts may include coordinatedata for the plurality of body parts, and may include a plurality of keypoints indicating a plurality of joint parts of the user included in theimage.

In order to acquire location data, an artificial intelligence model (notshown) for extracting a point of a human joint from an image may beused. In this case, the artificial intelligence model (not shown) may bestored in the memory 130.

In other words, the processor 140 may input the photographed image intoan artificial intelligence model (not shown) to acquire location datafor a plurality of body parts of the user included in the photographedimage, and input the acquired location data into the artificialintelligence model 131.

However, this is only an example, and the location data may be acquiredin various ways.

In addition, the processor 140 may identify a posture corresponding tothe training image. In this case, a metadata for the training image mayinclude information on a posture performed by the trainer included inthe training image for each time of the training image.

In this case, the processor 140 may determine a posture performed by thetrainer in the training image using metadata about the training image.

In addition, the processor 140 may identify the user's posture includedin the photographed image. For example, the processor 140 may identifythe user's posture included in the photographed image based on aprobability output from the artificial intelligence model 131.

In this case, the processor 140 may identify a posture having aprobability greater than a predetermined threshold value as the user'sposture.

For example, when a probability corresponding to the “squat” postureoutput from the artificial intelligence model 131 is greater than thepredetermined threshold, the processor 140 may identify the user'sposture as “squat”.

In this case, the processor 140 may identify that the user's posturematches the posture corresponding to the training image.

As another example, when a probability corresponding to the “incorrectlunge” posture output from the artificial intelligence model 131 isgreater than the predetermined threshold, the processor 140 may identifythat the user's posture has a probability of “incorrect lunge”.

In this case, the processor 140 may identify that the user's posturedoes not match the posture corresponding to the training image.

The processor 140 may control the display 120 to display a trainingguide based on whether the user's posture matches the posturecorresponding to the training image.

For example, if the user's posture is identified as matching the posturecorresponding to the training image, the processor 140 may display atraining guide including information indicating that the user's posturematches the posture corresponding to the training image.

For example, as shown in FIG. 5A, the processor 140 may control thedisplay to display a training image 510 in one area of the display 120and an image 520 of the user in another area of the display 120.

In this case, if the user's posture is identified as matching theposture corresponding to the training image, the processor 140 maydisplay a training guide 530 including text such as “correct squatposture” on the display 120.

According to an embodiment, the processor 140 may control the display todisplay information on an accuracy indicating a degree to which theuser's posture matches the posture corresponding to the training imageon the display 120.

In this case, the accuracy may be determined based on a probability thatthe user's posture corresponds to a specific posture.

For example, as shown in FIG. 5B, the processor 140 may display on thedisplay 120 a training guide 540 including texts such as “correct squatposture” and “accuracy: 98 points”.

When the user's posture is identified as matching the posturecorresponding to the training image, the processor 140 may control thedisplay 120 to identify information on a body part that has an exerciseeffect based on information on the body part output from the artificialintelligence model 131.

In other words, the artificial intelligence model 131 may outputinformation on a body part having an exercise effect on the identifieduser's posture. In this case, the processor 140 may control the displayto display information on the body part where the exercise effectappears based on information output from the artificial intelligencemodel 131.

For example, if the user's posture is identified as the “squat” posture,the artificial intelligence model 131 may output “good for legs, hipsand inner thighs” as information on the body part that have an exerciseeffect by the “squat” posture.

Accordingly, as shown in FIG. 6A, when the user's posture is identifiedas the “squat” posture, the processor 140 may control the display todisplay a training guide 610 including text such as “good for legs, hipsand inner thighs”.

As another example, the processor 140 may highlight a portion effectivefor exercise and control the display to display it on the display 120.

For example, as shown in FIG. 6B, the processor 140 may highlight 620the user's legs, hips, and inner thighs included in the photographedimage to indicate that the corresponding body part that would have anexercise effect.

If the user's posture is identified as not matching the posturecorresponding to the training image, the processor 140 may control thedisplay to display a training guide including information indicatingthat the user's posture does not match the posture corresponding to thetraining image.

For example, as shown in FIG. 7, the processor 140 may control thedisplay to display a training image 710 in one area of the display 120and an image 720 of the user in another area of the display 120.

In this case, if the user's posture is identified as not matching theposture corresponding to the training image, the processor 140 maycontrol the display to display a training guide 730 including text suchas “Incorrect lunge posture” on the display 120.

If the user's posture is identified as not matching the posturecorresponding to the training image, the processor 140 may control thedisplay 120 to display information on a body part that has a negativeexercise effect based on the information on the body part output fromthe artificial intelligence model 131.

In other words, the artificial intelligence model 131 may outputinformation on a body part that has a negative exercise effect for theidentified user's incorrect posture. In this case, the processor 140 maydisplay the information on the body part that have a negative exerciseeffect based on the information output from the artificial intelligencemodel 131.

For example, if the user's posture is identified as being the “incorrectlunge” posture, the artificial intelligence model 131 may output “waist”as information on the body part that have a negative exercise effect bythe “incorrect lunge” posture.

Accordingly, as shown in FIG. 8A, when the user's posture is identifiedas being the “incorrect lunge” posture, the processor 140 may display atraining guide 810 including text such as “posture that strains waist”on the display 120.

As another example, the processor 140 may highlight a part that has anegative effect on exercise on the display 120.

For example, as shown in FIG. 8B, the processor 140 may highlight 820the user's waist included in the photographed image to show that it is apart that may have a negative effect on the corresponding body part.

As described above, according to various embodiments of the disclosure,the display apparatus 100 may identify the user's posture and provide atraining guide according to whether the user's posture matches thetrainer's posture included in the training image.

FIG. 9 is a block diagram illustrating an example configuration of adisplay apparatus according to various embodiments.

Referring to FIG. 9, the display apparatus 100 may include a camera 110,a display 120, a memory 130, a processor (e.g., including processingcircuitry) 140, a communicator 150 (e.g., including communicationcircuitry), a speaker 160, and a user inputter (e.g., including inputcircuitry) 170. The components may be controlled by the processor 140.

The components shown in FIG. 9 are only examples, and at least somecomponents may be omitted or other components may be added according toembodiments.

In addition, since the camera 110, the display 120, the memory 130, andthe processor 140 have been described with reference to FIGS. 1 to 8,detailed descriptions of overlapping parts may not be repeated here.

The communicator 150 may include various communication circuitry forperforming communication with the external apparatus. For example, thecommunicator 150 may communicate with a web server (not shown) through anetwork.

For this operation, the communicator 150 may include variouscommunication circuitry included in various modules for accessing anetwork, such as a network card.

As another example, the communicator 150 may access a network using aWi-Fi communication module for performing Wi-Fi communication, andaccess the network through various mobile communication methods such as3G, long term evolution (LTE), 5G, or the like.

In this case, the processor 140 may access a server (not shown) thatprovides various applications through the communicator 150 to downloadapplications. For example, the processor 140 may access a server (notshown) through the communicator 150 to download a training application.

In addition, the processor 140 may access a server (not shown) throughthe communicator 150 to download the artificial intelligence model 131.

The speaker 160 may output various sounds. For example, the speaker 160may output audio corresponding to the training image.

The user inputter 170 may include various input circuitry for receivingvarious user commands For example, the user inputter 170 may include atouch panel or the like, and may also receive a user command from aremote control for controlling the display apparatus 100.

In this case, the processor 140 may control other components to executevarious functions according to a user command.

For example, when a user command for executing a training application isinput, the processor 140 may execute a training application stored inthe memory 130.

In addition, the processor 140 may display the training image on onearea of the display 120 and display the image photographed by the camera110 on another area of the display 120. In this case, the processor 140may output audio corresponding to the training image through the speaker160.

The processor 140 may identify the user's posture included in the imagephotographed through the camera 110 and provide a training guideaccording to whether the user's posture matches the posture of thetrainer included in the training image.

FIG. 10 is a flowchart illustrating an example method of controlling adisplay apparatus according to various embodiments.

A training image and an image photographed by the camera are displayed(S1010).

Location data on a plurality of body parts of the user included in thephotographed image is input into the artificial intelligence model toidentify the user's posture included in the photographed image (S1020).

The artificial intelligence model may be a model trained to identify theuser's posture based on location data for the plurality of body parts ofthe user included in the images and additional location data acquiredbased on the location data. In this case, the additional location datamay be acquired by rotating the location data.

A training guide may be displayed based on whether the user's posturematches the posture corresponding to the training image (S1030).

For the operation of S1030, if the user's posture is identified asmatching the posture corresponding to the training image, a trainingguide including information indicating that the user's posture matchesthe posture corresponding to the training image may be displayed, and ifthe user's posture is identified as not matching the posturecorresponding to the training image, a training guide includinginformation indicating that the posture of the user does not match theposture corresponding to the training image may be displayed.

In addition, the artificial intelligence model may be trained toidentify the user's first posture based on location data for a pluralityof body parts of the user in the first posture included in the imagesand additional location data acquired based on the location data.

The first posture may be a posture that matches a posture correspondingto the training image.

In addition, if the artificial intelligence model may be trained toidentify the body part that has a negative exercise effect according tothe first posture, and for the operation of S1030, if the user's postureis identified as matching the posture corresponding to the trainingimage, information on the body part that has an exercise effect may bedisplayed.

The artificial intelligence model may be trained to identify the user'ssecond posture based on location data for a plurality of body parts ofthe user in the second posture included in the images and additionallocation data acquired based on the location data.

The second posture may be a posture that does not match a posturecorresponding to the training image.

In addition, the artificial intelligence model may be trained toidentify body parts that have negative exercise effects according to thesecond posture, and for the operation of S1030, if the user's posture isidentified as not matching the posture corresponding to the trainingimage, information on the body part that has a negative exercise effectmay be displayed based on the information on the body part output fromthe artificial intelligence mode.

An example method of identifying the user's posture using the artificialintelligence model and providing guide information accordingly has beendescribed.

According to an embodiment, the various embodiments described above maybe implemented as software including instructions stored in amachine-readable storage media which is readable by a machine (e.g., acomputer). The device may include the electronic device according to thedisclosed embodiments, as a device which calls the stored instructionsfrom the storage media and which is operable according to the calledinstructions. When the instructions are executed by a processor, theprocessor may directory perform functions corresponding to theinstructions using other components or the functions may be performedunder a control of the processor. The instructions may include a codemade by a compiler or a code executable by an interpreter. Themachine-readable storage media may be provided in a form of anon-transitory storage media. The ‘non-transitory’ storage media doesnot include a signal and is tangible, and does not distinguish whetherdata is stored semi-permanently or temporarily in the storage media.

In addition, according to an embodiment, the methods according tovarious embodiments described above may be provided as a part of acomputer program product. The computer program product may be tradedbetween a seller and a buyer.

The computer program product may be distributed in a form of themachine-readable storage media (e.g., compact disc read only memory(CD-ROM) or distributed online through an application store (e.g.,PlayStore™). In a case of the online distribution, at least a portion ofthe computer program product may be at least temporarily stored orprovisionally generated on the storage media such as a manufacturer'sserver, the application store's server, or a memory in a relay server.

While the disclosure has been illustrated and described with referenceto various example embodiments, it will be understood that the variousexample embodiments are intended to be illustrative, not limiting. Itwill be further understood by those skilled in the art that variouschanges in form and detail may be made without departing from the truespirit and full scope of the disclosure, including the appended claimsand their equivalents.

What is claimed is:
 1. A display apparatus comprising: a camera; adisplay; a memory storing an artificial intelligence model trained toidentify a posture of a user based on location data with respect to aplurality of body parts of a user included in images and additionallocation data acquired based on the location data; and a processorconfigured to: control the display to display a training image andimages photographed by the camera, identify the posture of the userincluded in the photographed images by inputting the location data withrespect to the plurality of body parts of the user included in thephotographed images, and control the display to display a training guidebased on whether the posture of the user matches a posture correspondingto the training image.
 2. The apparatus of claim 1, wherein theadditional location data includes data acquired by rotating the locationdata.
 3. The apparatus of claim 1, wherein the processor is configuredto: based on the posture of the user matching the posture correspondingto the training image, control the display to display the training guideincluding information indicating that the posture of the user matchesthe posture corresponding to the training image, and based on theposture of the user not matching the posture corresponding to thetraining image, control the display to display the training guideincluding information indicating that the posture of the user does notmatch the posture corresponding to the training image.
 4. The apparatusof claim 1, wherein the artificial intelligence model is configured,based on location data with respect to a plurality of body parts of theuser in a first posture included in the images and additional locationdata acquired based on the location data, to be trained to identify thefirst posture of the user, and wherein the first posture matches theposture corresponding to the training image.
 5. The apparatus of claim4, wherein the artificial intelligence model is configured to be trainedto identify a body part that has an exercise effect based on the firstposture, and wherein the processor is configured to, based on theposture of the user matching the posture corresponding to the trainingimage, control the display to display information on the body part thathas the exercise effect based on information on a body part output fromthe artificial intelligence model.
 6. The apparatus of claim 1, whereinthe artificial intelligence model is configured to be trained toidentify a second posture of the user based on location data withrespect to a plurality of body parts of the user in a second postureincluded in the images, and additional location data acquired based onthe location data, and wherein the second posture does not match theposture corresponding to the training image.
 7. The apparatus of claim6, wherein the artificial intelligence model is configured to be trainedto identify a body part that has a negative exercise effect based on thesecond posture, and wherein the processor is configured to, based on theposture of the user not matching the posture corresponding to thetraining image, control the display to display information on a bodypart that has the negative exercise effect based on information on abody part output from the artificial intelligence model.
 8. A method forcontrolling a display apparatus comprising: displaying a training imageand images photographed by a camera; identifying a posture of a userincluded in the photographed images by inputting a location data withrespect to a plurality of body parts of a user included in thephotographed images; and displaying a training guide based on whetherthe posture of the user matches a posture corresponding to the trainingimage, wherein an artificial intelligence model is configured to betrained to identify the posture of the user based on the location datawith respect to the plurality of body parts of the user and additionallocation data acquired based on the location data.
 9. The method ofclaim 8, wherein the additional location data includes data acquired byrotating the location data.
 10. The method of claim 8, wherein thedisplaying comprises, based on the posture of the user matching theposture corresponding to the training image, displaying the trainingguide including information indicating that the posture of the usermatches the posture corresponding to the training image, and based onthe posture of the user not matching the posture corresponding to thetraining image, displaying the training guide including informationindicating that the posture of the user does not match the posturecorresponding to the training image.
 11. The method of claim 8, whereinthe artificial model is configured, based on location data with respectto a plurality of body parts of the user in a first posture included inthe images and additional location data acquired based on the locationdata, to be trained to identify the first posture of the user, andwherein the first posture matches the posture corresponding to thetraining image.
 12. The method of claim 11, wherein the artificialintelligence model is configured to be trained to identify a body partthat has an exercise effect based on the first posture, and wherein thedisplaying comprises, based on the posture of the user matching theposture corresponding to the training image, displaying information onthe body part that has the exercise effect based on information on abody part output from the artificial intelligence model.
 13. The methodof claim 8, wherein the artificial intelligence model is configured tobe trained to identify a second posture of the user based on locationdata with respect to a plurality of body parts of the user in a secondposture included in the images and additional location data acquiredbased on the location data, and wherein the second posture does notmatch the posture corresponding to the training image.
 14. The method ofclaim 13, wherein the artificial intelligence model is configured to betrained to identify a body part that has a negative exercise effectbased on the second posture, and wherein the displaying comprises, basedon the posture of the user not matching the posture corresponding to thetraining image, displaying information on a body part that has thenegative exercise effect based on information on a body part output fromthe artificial intelligence model.